Papers with fine-tuned systems
QUDeval: The Evaluation of Questions Under Discussion Discourse Parsing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics poorly approximate parser quality, says a new study . questions under discussion is a linguistic framework that views discourse as asking questions and answering them . |
| Approach: | They propose a framework for automatic evaluation of QUD parsing . they use a dataset of fine-grained evaluation of 2,190 QUD questions . |
| Outcome: | The proposed framework shows that satisfying constraints of QUD is still challenging for modern LLMs. |
Background Summarization of Event Timelines (2023.emnlp-main)
Copied to clipboard
| Challenge: | Generating concise summaries of news events is a challenging task for newcomers to a news story. |
| Approach: | They propose a task of background news summarization that complements each timeline update with a background summary of relevant preceding events. |
| Outcome: | The proposed system performs well on a question-answering-based evaluation metric, Background Utility Score (BUS). |